Retailers have been collecting large quantities of point-of-sale data in many different industries. One area that has been particularly active in terms of collecting this type of data is grocery retailing. Loyalty card programs at many grocery chains have resulted in the capture of millions of transactions and purchases directly associated with the customers making them.
Despite this wealth of data, the perception in the grocery industry is that this data has been of little use. The data collection systems have been in place for several years but systems to make sense of this data and create actionable results have not been very successful. There have been efforts to utilize the retail transaction data. For example, research in mining association rules (R. Agrawal and R. Srikant, Fast algorithms for mining association rules. In Proc. of 20th Int'l Conference on Very Large Data Bases, Santiago, Chile, 1994) has led to methods to optimize product assortments within a store by mining frequent item-sets from basket data (T. Brijs, G. Swinnen, K. Vanhoof, and G. Wets, Using association rules for product assortment decisions: A case study. In Knowledge Discovery and Data Mining, pages 254-260, 1999). Customer segmentation has been used with basket analysis in the direct marketing industry for many years to determine which customers to send mailers to. Additionally, a line of research based on marketing techniques developed by Ehrenberg (A. Ehrenberg, Repeat-Buying: Facts, Theory, and Applications, Charles Griffin & Company Limited, London, 1988) seeks to use a purchase incidence model with anonymous data in a collaborative filtering setting (A. Geyer-Schulz, M. Hahsler, and M. Jahn, A customer purchase incidence model applied to recommender systems, in WebKDD2001 Workshop, San Francisco, Calif., August 2001).
Traditionally, most of the data mining work using retail transaction data has focused on approaches that use clustering or segmentation strategies. Each customer is “profiled” based on other “similar” customers and placed in one (or more) clusters. This is usually done to overcome the data sparseness problem and results in systems that are able to overcome the variance in the shopping behaviors of individual customers, while losing precision on any one customer.
A major reason that individually targeted applications have not been more prominent in retail data mining research is that in the past there has been no effective individualized channel to the customer for brick & mortar retailers. Direct mail is coarse-grained and not very effective as it requires the attention of customers at times when they are not shopping and may not be actively thinking about what they need. Coupon based initiatives given at checkout-time are seen as irrelevant as they can only be delivered after the point of sale. Studies have shown that grocers lose out on potentially 11% of sales due to forgotten items, which highlights the need to find effective individual channels to customers at the point of sale prior to check out.
With the advent of PDA's and shopping cart mounted displays, such as the model Symbol Technologies is piloting with a New England grocer, retailers are in a position now to deliver personalized information to each customer at several points in the store. In fact, a few systems have been developed and attempt to deliver personalized information to customers. For example, the IBM Easi-Order system allows a list to be developed on a customer's PDA, which is then sent to the store to be compiled and picked up. (R. Bellamy, J. Brezin, W. Kellogg, and J. Richards, Designing an e-grocery application for a palm computer: Usability and interface issues, IEEE Communications, 8(4), 2001). In a system developed at Georgia Tech, a PDA was used as a shopping aide during a shopping trip to show locations and information on items in a list (E. Newcomb, T. Pashley, and J. Stasko, Mobile computing in the retail arena, in Proceedings of the conference on Human factors in computing systems (CHI2003), pages 337-344. ACM Press, 2003). In each of the IBM and Georgia Tech systems, the shopping list was emphasized as the essential artifact of a grocery trip, enabling all other interactions. Both also stated as a design goal that it should be possible to compile or augment a shopping list per customer based on previous purchase history. In another example, the 1:1Pro system was designed to produce individual profiles of customer behavior in the form of sets of association rules for each customer which could then be restricted by a human expert (G. Adomavicius and A. Tuzhilin, Using data mining methods to build customer profiles, IEEE Computer, 34(2):74-82, 2001). Despite these efforts, there has not been a thorough experimental attempt to predict and evaluate individually personalized customer shopping lists from transactional data with a large set of customers.
Therefore, given the massive amounts of data presently being captured, and the imprecise predictive ability of clustering and segmentation approaches, there is a need to better utilize the captured data, such as a better prediction of a shopping list. Likewise, there is a need for a system to provide a predictive shopping list to customers using the consumer models using reduced processor resources to be able to deliver the lists locally on mobile processing devices attached to shopping carts. Also, there is a need to better utilize the captured data to provide enhanced promotion and planning for retail establishments and others in the supply chain.